1. Field of the Invention
This invention relates to an encoding method and apparatus for compressing picture/speech signals for recording the signals in this compressed form on a recording medium. More particularly, it relates to an encoding method and apparatus used with advantage for storing picture/speech signals on so-called package mediums, such as digital video discs (DVDs).
2. Description of the Related Art
In an encoding system for compressing and encoding the video signals for compression for recording on a so-called package medium, such as a digital video disc (DVD) or a video CD, there is routinely used such a method in which the difficulty of encoding of a video material is first measured and bit assignment is then performed, in terms of a frame of each video information as a unit, so that the bits assigned to each frame will be within a given number of bytes comprised in the range of the recording capacity of the package medium based on the measured value of the encoding difficulty. This two-stage encoding processing is hereinafter referred to as a two-path encoding.
FIG. 1 shows a basic structure of an authoring system used for authoring for a digital video disc (DVD) in which the video information is encoded for compression.
In this authoring system, various units used for respective encoding steps, namely a video encoder 22, an audio encoder 21, a sub-title encoder 23, a menu encoder 24, an emulator 26 and a multiplexer 25, a hard disc array (RAID) 4 in which to write the encoding results, and a supervisor 3 for supervising the operation of these units, are interconnected by a network 2.
This authoring system is superior in operating efficiency because the encoding operations for respective parts, such as video or audio, can be performed substantially in parallel.
FIG. 2 shows an illustrative structure of a conventional encoding system in the authoring system shown for example in FIG. 1.
The supervisor 103 controls the video encoding system in its entirety and sets encoding conditions for the respective encoding systems, such as video, audio or menu, while receiving reports on the results of encoding. In the present case, the video encoding conditions are specified by a file ┌v. enc┘ and a report is made from the video encoder side on an address ┌v. adr┘ on a RAID 104 on which a bitstream on the encoding results is written for the video encoder side and on data ┌vxxx. aui┘ required for bitstream multiplexing.
A main controller 111 controls the operation of this video encoding system in its entirety by communication with the supervisor 103 connected to the main controller 111 over the network 102.
Specifically, the main controller 111 accepts the control from the supervisor 103 and the operation by an operator, under control by a graphical user interface (GUI) 114, in order to control the operation of the encoder 112 and a video tape recorder (VTR) 110 by a bit assignment unit 115, an encoder control unit 116 and a VTR control unit 117. This permits the main controller 111 to encode the material being processed, in accordance with the encoding conditions as notified by the supervisor 103, to notify the results of the processing to the supervisor 103. The main controller 111 also accepts the setting byte operator via the GUI unit 114 to modify the detailed conditions for encoding.
The GUI unit 114 of the main controller 111 controls three programs, namely a bit assignment program ┌BIT_ASSIGN┘, an encoder control program ┌CTRL_ENC┘ of an encoder control program and a VTR control program of the VTR control unit 117.
The bit assignment unit 115 determines, on the frame basis, the conditions for encoding processing in accordance with the encoding conditions file ┌v. enc┘ notified from the supervisor 103 to notify he control data under these conditions to the control unit 116 in accordance with the file form ┌CTL file┘.
The bit assignment unit 115 then sets bit assignment in the encoding processing and modifies the so-set conditions depending on the operator""s actuations. When the compressed video data D2 is recorded on the RAID 104, the bit assignment unit 115 notifies the address data ┌v. adr┘ on the RAID 104, having the video data D2 written therein, along with the information ┌vxxx. aui┘ such as the data quantity required in a downstream side multiplexing operation, to the supervisor 103.
An encoder control unit 116 controls the operation of the encoder 112 in accordance with the control file ┌CTL file┘ notified from the bit assignment unit 115. The encoder control unit 116 notifies data on encoding difficulty ┌Difficulty┘ to the bit assignment unit 115 on the frame basis. When the video data D2 is recorded on the RAID 104, the recording address data ┌v. adr┘ and data required for subsequent multiplexing ┌vxxx. aui┘ is notified to the bit assignment unit 115.
The VTR control unit 117 controls the operation of the video tape recorder (VTR) 110 in accordance with the editing list notified from the supervisor 3 to reproduce the material of a desired editing subject.
The video tape recorder (VTR) 110 reproduces the video data D1 recorded on the magnetic tape to output the reproduced data to the encoder 112 in accordance with the editing list communicated from the supervisor 103 via the main controller 111.
The encoder 112 has its operation switched in accordance with the conditions notified from the supervisor 103 via main controller 111 to encode the video data D1 outputted by the VTR 110 by compression in accordance with the MPEG (Moving Pictures Experts Group) technique.
The encoder 112 at this time notifies the results of the encoding processing to the main controller 111 which then controls the encoding conditions for the data compression to control the quantity of generated bits. This permits the main controller 111 to grasp the amount of bits generated on data compression on the frame basis.
At the time of preliminary setting the encoding conditions for two-path encoding, the encoder 112 simply compresses video data D1 to notify the results of processing to the main controller 111. However, at the time of ultimate data compression processing, that is encoding per se, compressed video data D2 is recorded on the RAID 104 and the addresses for the recorded data and the data quantity etc are notified to the main controller 111.
A monitor device 113 is configured for monitoring the video data D2 compressed by the encoder 112. This video encoding system is able to perform so-called pre-view of ascertaining the results of data compression by an operator with the aid of the monitor 113 as the occasion may demand. The operator actuates the main controller 111 based on the results of the preview to modify the details of the encoding conditions.
The MPEG is a system of compressing data by removing redundancy in the time domain by motion compensated prediction. In this MPEG system, there are used three sorts of the encoded pictures, namely an intra-picture (I-picture) encoded within one frame, a predictive picture or a P-picture, encoded on predicting the present from a past picture, and a bidirectionally predicted picture (B-picture) encoded on prediction from a past picture and a future picture.
These pictures are grouped into a unit termed a group-of-pictures GOP in which there is contained at least one I-picture.
The conventional two-path encoding operation is explained with reference to the structure of a video encoding system shown for example in FIG. 1.
FIG. 3 shows a basic processing sequence for two-path encoding in the conventional video encoding system shown in FIG. 2.
First, at step S51, the encoding conditions ┌v. enc┘, such as the total bit quantity or the maximum rate, assigned to the video information, via the network 102 from the supervisor 103. The encoder control unit 116 is set in accordance with these encoding conditions.
Then, at step S52, the encoder control unit 116 measures the encoding difficulty (Difficulty) of the material for encoding, using the encoder 112. At this time, the dc values or the motion vector quantity ME of each pixel in the material are measured simultaneously. At this time, the dc value of each pixel and the motion vector magnitude ME are measured simultaneously. A file is formulated on the basis of the measured results.
The encoding difficulty is actually measured in the following manner.
The video material, as an encoding material, is reproduced from a digital video cassette as a master tape by the VTR 110.
An encoder control unit 116 measures the encoding difficulty of the video information D1 reproduced by the VTR 111 via encoder 112. Here, the number of quantization steps is set to a fixed value, at the time of encoding, to measure the quantity of generated bits. In a picture having abundant movements and hence high frequency components, the amount of generated bits is increased, whereas, in a still picture or a picture having abundant flat portions, the amount of generated bits is decreased. This magnitude of the generated bit quantity is a measure of the encoding difficulty.
At the next step S53, the encoder control unit 116 executes a bit assignment calculating program ┌BIT_ASSIGN┘ in the bit assignment unit 115, depending on the magnitude of the encoding difficulty of each picture as measured at step S52 in accordance with the encoding conditions as set at step S51, to effect distribution calculations of the quantity of assigned bits (target quantity).
Using the results of the above-mentioned bit assignment calculations, a provisional encoding is carried out in order to permit the user to judge at step S54, whether or not the ultimate encoding is to be carried out, depending on the picture quality of an output of a local decoder enclosed in the encoder 112. In actuality, the bit stream by the above-mentioned bit assignment is not outputted to the RAID 4 and the picture quality is ascertained in the preview mode in which the user can freely specify a desired processing range.
At step S55, the picture quality is evaluated. If the picture quality is found to be objectionable (NG), processing transfers to step S56 to perform a customizing operation for picture quality adjustment, such as raising the bitrate of the objectionable portion or adjusting the filter level. At the next step S57, bit assignment is re-calculated.
Then, processing reverts to step S54 to make preview of the customized portion. At the next step S55, the picture quality is ascertained. If the picture quality of all portions is good, processing transfers to step S58 to perform ultimate encoding processing (encoding per se) for the entire material by the bit assignment re-calculated at step S57.
If at step S55 the picture quality is found to be unobjectionable, processing directly transfers to step S58 to execute ultimate encoding by the bit assignment calculated at step S53.
At step S59, post-processing, such as writing a bitstream as the result of encoding via small computer system interface (SCSI) in the RAID 104, to terminate the two-path encoding processing.
After execution of the encoding at step S58, the video encoder control unit 116 reports the information on the results of encoding as described above to the supervisor 103 over the network 102.
Meanwhile, the processing of the respective steps excluding steps S52, S54 and S58 is executed off-line.
If, in the above-described sequence of the processing operations, the encoding material cannot be recorded in a sole video tape, it is recorded across plural rolls of tape. Due to the tape exchange operations, encoding cannot be executed continuously.
In the case of a multi-angle portion, the time code of this portion is the same between different angle blocks. Therefore, in this case, continuous encoding again is not possible. The processing unit, which entails transient interruption of the encoding operation, is termed an encoding unit.
The bit allocation calculations in the above-described two-path encoding are hereinafter explained.
First, the total bit quantity ┌QTY_BYTES┘ and the maximum bitrate ┌MAXRATE┘, of the recording capacity of the package medium storing the encoding results, assigned to the video, are designated from the authoring system. On the other hand, the total number of bits, limited so as to be not larger than the maximum bitrate ┌USB_BYTES┘, is found. Then, ┌SUPPLY_BYTES┘, as a target value of the total number of targets, is calculated from the total number of frames and a value corresponding to ┌USB_BYTES┘ less the number of bits ┌TOTAL_HEADER┘ required for ┌GOP header┘, is calculated. The quantity of bits assigned to the respective pictures is assigned so as to be not larger than this ┌SUPPLY_BYTES┘.
FIG. 4 shows a typical instance of the processing operations for calculating the bit assignment in the step S53 shown in FIG. 3.
At step S61, the total bit quantity ┌QTY_BYTES┘ and the maximum bitrate ┌MAXRATE┘ sent from the supervisor as described above are entered.
Then, at step S62, the file showing the measured results of the encoding difficulty ┌Difficulty┘, prepared at step S52 of FIG. 3, is read.
Then, at step S63, a scene changing point is detected from variations in the parameters such as the dc values of the respective pictures and the magnitudes of the motion vectors ME, measured simultaneously with measurement of the encoding difficulty.
FIG. 5 shows the processing of frames designated for scene changes.
From the variations in the parameters, such as dc values of the respective pictures or magnitudes of the motion vectors, measured simultaneously with measurement of the encoding difficulty, it is possible to detect the scene changing point. This renders it possible to change the P-picture, detected as a scene change, into an I-picture to improve picture quality, as shown in FIG. 5. The number of frames N is herein set to 15 (N=15).
At the next step S64, chapter boundary processing is executed. During chapter search by the disc reproducing device, reproduced pictures jump from non-specified pictures. In order to prevent a reproduced picture from being disturbed eve in such case, the picture type is changed or the GOP length limited by this chapter boundary processing so that the chapter position will always be at the leading position of the GOP.
FIG. 6 shows the processing of the boundary of this chapter-designated frame, that is the cell boundary.
At step S65, the value of the encoding difficulty is interpolated/corrected in meeting with the picture types, such as I-, P- or B-pictures, as modified as a result of the above-described sequence of operations.
This processing is done because the maximum number of displayed fields at the time of decoding of each GOP is limited in the case of a recording medium, such as DVD, and hence it may be an occurrence that the GOP length exceeds the above limitation as a result of the change in the GOP structure accompanying the change in the picture type. In such case, the P-picture is changed to the I-picture to reduce the GOP length in order to meet the limitation by way of GOP limitation.
At the next step S66, the bit quantity is assigned from one encoding unit (ENCU) to another.
Then, at step S67, bit assignment (supply_bytes[encu_nb]) is made from one ENCU to another, depending on the encoding difficulty obtained by the interpolation/correction at step S65 and the number of bits ┌SUPPLY_BYTES┘ given to the entire encoding material. This bit assignment is the target value of the sum of the target bits. Then, picture-based bit assignment is executed in dependence upon the target value.
At step S68, the address of the RAID at the time of writing a bitstream as the encoded result is calculated. At step S69, a control file for an encoder is formulated at step S69 to terminate a sequence of the processing operations.
By the above processing, the target number of bits is calculated from one picture to another to formulate a control file for an encoder responsive to the encoding difficulty of the material and the number of bits ┌SUPPLY_BYTES┘ accorded to the material in its entirety.
Referring to an instance of bit assignment in case the material contains so-called angle blocks, shown in FIG. 7, the above-described sequence of bit assignment is explained hereinafter in further details. The multi-angle will be explained in detail subsequently.
FIG. 8 shows the basic sequence of bit allocation which takes the weighting coefficients for seamless angle portions into account.
First, at step S71, the total bit quantity ┌QTY_BYTES┘ of the recording capacity portion of the package medium assigned to the video is first acquired at step S71 from the supervisor.
Next, at step S72, the total number of bits ┌USB_BYTES┘ designated as the encoding condition is found with respect to the total number of bits ┌QTY_BYTES┘:
USB_BYTES=min(QTY_BYTES, MAXRATExc3x97KTxc3x97total_frame_number)xe2x80x83xe2x80x83(1).
If the format of the video material is of the NTSC system or of the PAL system, KT=xe2x85x9 (bits)/30 Hz and K=xe2x85x9 (bits)/25 Hz, respectively. Meanwhile, ┌total_frame_number┘ is the total number of frames of the materials for encoding and ┌min(s, t)┘ is a function which selects a smaller one of s or t.
Then, at step S73, the ┌USB_BYTES┘ less the total of the number of bits ┌TOTAL_HEADER┘ required for ┌GOP header┘ or ┌xcexa3TOTAL_SUPPLY_BYTES┘ is found. The total xcexa3 means the sum for the respective recording layers.
At step S74, ┌ALL_ANGLE_BYTES┘ of the seam angle portion is calculated from the total number of frames ┌TOTAL_FRAMEB┘. The reason the bits are first assigned to the seamless angle block is that more strict limitations are placed on the seamless angle block than on other portions so that a larger number of bits are required for the seamless angle block, while the same amount of bits needs to be assigned to equivalent seamless angle block s.
If the number of frames of each encoding unit (ENCU) is ┌ENCU_frame[encu_nb}┘,
supply_bytes[encuxe2x80x94nb]=(USB_BYTESxe2x88x92TOTAL_HEADER)xc3x97ENCU_frame[encuxe2x80x94nb]/total_framebxe2x80x83xe2x80x83(2)
ALL_ANGLE_BYTES=xcexa3supply_bytes (seamless angle block)xe2x80x83xe2x80x83(3).
Since the numbers of frames of the equivalent seamless angle blocks are equal, the corresponding ┌supply_bytes┘ are necessarily equivalent.
Then, at step S75, the ┌supply_bytes┘ of each ENCU of the portion other than the angle block portion, that is ┌ALL_SUPPLY_BYTES┘, are calculated with respect to a value equal to ┌USB_BYTES┘ less ┌ALL_ANGLE_BYTES┘. If bit assignment is done based simply on the frame number ratio, optimum bit assignment cannot be achieved in case picture difficulty varies from one ENCU to another. Therefore, the sum total of the encoding difficulty of each ENCU is found and, at step S76, the bit quantity assignment is done at the ratio therebetween.
There are occasions wherein the user is desirous to act consciously on the bit assignment among the ENCUs at the material stage. For example, in inserting commercial ads before a cinema material, it may be desired to raise the bit rate consciously only for this portion. This desideratum may be realized by setting the weighting coefficient ┌e_weight┘ for the sum total of the encoding difficulty of each ENCU.
For example, this weighting coefficient may be set by a file shown in the following table 1 and loaded at the time of executing bit calculations:
DIFF_SUM=xcexa3ENCU_diff[encuxe2x80x94nb](ENCU portion for other than seamless angle portion)xe2x80x83xe2x80x83(4).
ALL_SUPPLY_BYTES=USB_BYTESxe2x88x92TOTAL_HEADERxe2x88x92ALL_ANGLE_BYTESxe2x80x83xe2x80x83(5).
Supply_bytes[encuxe2x80x94nb]=ALL_SUPPLY_BYTES*ENCU_diff[encuxe2x80x94nb]/DIFF_SUMxe2x80x83xe2x80x83(6). 
In the following explanation, it is assumed that bit quantities are assigned on the GOP basis and subsequently the bits are assigned in each GOP in meeting with the difficulty of each picture ┌GOP-DIFFICULTY┘. The GOP-based quantity of bit assignment at the time of encoding ┌GOP_target┘ is assigned in dependence upon the sum of the GOP-based difficulty ┌GOP_target┘.
FIG. 9 shows an example of the simplest function for converting the sum of the GOP-based encoding difficulty ┌gop_diff┘ and the GOP-based amount of bit assignment ┌gop_target┘.
Here, with ┌gop_target┘ being Y, ┌gop_diff┘ being X and with DIFFICULTY_SUM=ENCU_diff[encu_nb], an evaluation function represented as
Y=AX+B
is used. Also, the sum total of ┌Difficulty┘ of all pictures or ┌ENCU_diff[encu_nb]┘ is used.
B=GOP_MINBYTES
xe2x80x83xcexa3y=Axc3x97xcexa3x+Bxc3x97n
where xcexa3y=supply_bytes[encu_nb], xcexa3X=ENCU_diff[encu_nb] and n is the sum total of the GOPs. Therefore,
A=supply_bytes[encuxe2x80x94nb]xe2x88x92Bxc3x97n)/ENCU_diff[encuxe2x80x94nb].
GOP_target=Axc3x97GOP_diff+Bxe2x80x83xe2x80x83(8).
Then, bit assignment is done in each GOP in meeting with the encoding difficulty of each picture ┌gen_bit[k]┘. This ┌gen_bit[k]┘ is a value of the encoding difficulty of a k""th frame as measured by provisional encoding, with the picture difficulty being higher the larger the value of this encoding difficulty ┌gen_bit[k]┘. If bit assignment to each picture in the GOP is rendered proportionate to the value of encoding difficulty, the target quantity in each picture is found by the following equation:
target(k)=GOP_TARGETxc3x97┌gen_bit[k]┘/GOP_diffxe2x80x83xe2x80x83(9)
where 1xe2x89xa6kxe2x89xa6the number of pictures in a GOP.
If, in this case, there is an extremely difficult picture in the material, that is a picture with a large value of ┌GOP_diff┘, the quantity of the ┌gop_target┘ is drastically increased such that the maximum rate allowed in the system is exceeded. It is therefore necessary to apply a limiter with a fixed quantity ┌GOP_MAXBYTES┘. The minimum target quantity is also limited by ┌GOP_MINBYTES┘.
For video encoding by the MPEG, it is mandatory to make bit assignment in consideration of the residual buffer volume of a virtual decoder. The calculations for the residual buffer quantity are termed video buffering verifier (VBV).
The method for calculating the virtual residual buffer quantity is hereinafter explained.
FIG. 10 shows the method for calculating the VBV.
If the recording medium is the so-called DVD, and the start point of a buffer for the k""th picture with the buffer size of ┌VBV MAX┘ of 1.75 Mbits is ┌Occupancy_up(k)┘, with the target quantity of the k""th picture being ┌target(k)┘, the residual buffer quantity ┌Occupancy_down(k)┘ after supplying the bits to the picture is represented by the equation (7).
In this buffer is stored the data quantity ┌SYSTEM_SUPPLY┘ of a bitrate corresponding to the video data quantity from a decoder pickup. The residual buffer quantity after this supply ┌Occupancy_up(k+1)┘ is represented by the equation (8).
The supply amount corresponds to the right upward rising quantity in the drawing. The larger the rate of the bits supplied, the larger becomes the gradient and the larger is the amount of data stored in the buffer. If the buffer is charged to its full capacity, the data supply from the pickup to the buffer ceases. Therefore, buffer overflow need not be taken into consideration. This means that it is unnecessary to perform control to just a pre-set value, it being only sufficient to perform control so that the pre-set value will be surpassed.
Conversely, if the data volume of each picture is large, the data stored in the buffer is decreased. The target bit quantity is calculated so that the residual buffer quantity will not be decreased to below a pre-set value. The first ┌Occupancy_up(0)┘ of these calculations is started from a fixed value, herein VBV MAXxc3x97⅔.
In the following, ┌Occupancy_up┘ and ┌Occupancy_down┘ mean upper and lower points of each picture on the graph, respectively.
Occupancy_up(0)=VBV MAXxc3x97⅔xe2x80x83xe2x80x83(10)
Occupancy_down(k)=Occupancy_up(k)xe2x88x92target(k)xe2x80x83xe2x80x83(11)
Occupancy_up(k+1)=Occupancy_down(k+1)+SYSTEM_SUPPLYxe2x80x83xe2x80x83(12)
SYSTEM_SUPPLY=MAXRATE(bps)xc3x97KTxe2x80x83xe2x80x83(13).
In the following, there is shown an instance of target bit allocation calculations on the GOP basis.
FIG. 11 shows an instance of target bit assignment in case VBV buffer calculations are made with respect to the target quantity found in consideration of the evaluation function and ┌GOP_MAXRATE┘ limitations.
The first, fourth and seventh pictures in FIG. 11 are lower in VBV value than the value of ┌VBV MIN┘ which is the lower limit of the VBV buffer. Therefore, the target quantity of the GOP containing the picture whose VBV value is decreased to below ┌VBV MIN┘ is diminished.
If the minimum value of ┌occupancy┘ when VBV calculations are executed on the target quantity prior to imposing the VBV limitations in the GOP is ┌Occ_min┘, the quantity of adjustment is given by
xcex3=┌Occ_startxe2x88x92VBV MIN)/(VBVSTARTxe2x88x92Occ_min) for Occupancy_min less than VBV MINxe2x80x83xe2x80x83(14)
where the start point for limitation ┌kstart┘ has a reference value ┌VBVLINE┘ which is the value for k not smaller than VBV MAX X (xc2xe), with the value of ┌Occupancy_up(k)┘ being ┌Occ_start┘, with
target(j)=target(j)xc3x97r(kstartxe2x89xa6jxe2x89xa6k)
for each target.
FIG. 12 shows an instance of target bit assignment following the above-described VBV limitation processing.
The first, fourth and seventh pictures, whose VBV were below ┌VBV MIN┘ as the lower limit of the VBV buffer prior to the processing for VBV limitation, are adjusted so that the VBV value will not be lower than this limit value.
By performing the encoding by a control file prepared using the target quantity, thus found, the bit assignment can be achieved in dependence upon the difficulty of the picture as a video material, so that a smaller number of bits can be assigned to a simpler picture while a larger number of bits can be assigned to a more intricate picture, thus realizing variable rate encoding suffering from less variations throughout the performance time.
Meanwhile, the disc format for a digital video disc (DVD) provides a disc having two recording layers on one side and a disc having a sum of four recording layers on its both sides.
How many number of bytes is to be assigned to the video data recorded on each of the respective recording layers is pertinent to the configuration of the entire disc and hence is specified by the supervisor as encoding conditions. When setting these encoding conditions on the part of the supervisor, difference in difficulty of pictures as the video material scheduled to be recorded on the respective recording layers is not known, such that there is no alternative but to distribute the bytes by having reliance simply on the proportions of frames recorded on the respective recording layers.
However, since the pictures of the materials recorded on the respective recording layers undergo variations, there is produced difference in picture quality between the different recording layers if the bytes are uniformly distributed in this manner among the recording layers.
For example, if the performance time for the first layer is equal to that for the second layer, the same number of bytes is specified as the encoding conditions. However, if a large number of simple still pictures are contained in the first layer, while a large number of pictures having a larger information volume are contained in the second layer, there is produced a significant difference in picture quality between the first and second layers, such that alien feeling is invoked if the two layers are switched during playback on a disc reproducing device.
FIGS. 13A and 13B show an instance of bit assignment in accordance with the conventional system on a recording medium such as a so-called DVD having two recording layers.
The encoding units ENCU1 and ENC2 are totally identified in the number of frames and in the sum of encoding difficulty and hence are of the same encoding conditions. However, the bit assignment for the recording layer ┌Layer 1┘ differs from that for the recording layer ┌Layer 2┘. If the encoding is done under this condition, there is produced an explicit picture quality difference between the encoding units ENCU1 and ENC2.
FIG. 14 collectively shows the results of bit assignment for the respective encoding units ENCU in the example shown in FIGS. 13A and 13B.
This difference among the different recording layers can be confirmed only by the evaluation of the picture quality following the encoding. Thus, the encoding conditions are changed at a time point when the picture quality is verified to be undesirable. However, it is not known to which extent the encoding conditions can be adjusted, so that the operation is a cut-and-try operation, thus significantly lowering the efficiency.
An instance in which the material for encoding contains a so-called angle block.
The angle reproduction in the digital video disc DVD is the reproduction enabling the user to reproduce a picture of the same object at the same time instant from plural angles, as shown in FIG. 15. The angle reproduction encompasses a non-seamless angle reproduction which allows for transient interruption of a playback picture at the time of switching, such as by black insertion, and seamless angle reproduction in which switching is allowed to occur without disturbance, that is seamlessly. Here, an instance of switching between three pictures from different angles, that is ┌Angle 1┘, ┌Angle 2┘ and ┌Angle 3┘, is shown.
FIG. 16 shows an instance of data containing angle blocks.
In this figure, the encoding units ENC2, ENCU4 and ENCU 5 denote angle combinations. In the case of the seamless angle, the number of pictures and the GOP structure are designed to be the same for the combination of angles, namely ENC2, ENCU4 and ENCU 5.
If the angle is switched during reproduction, the playback pickup of the reproducing device is moved to a position of data at a predetermined angle to start readout of data. For reducing the delay required for angle switching at the time of reproduction, it is necessary for the distance of this movement to be small. The angle block data has the positions on the disc interleaved in terms of M GOPs as a unit, with the value of M being different depending on the encoding conditions.
An instance of this interleaving is shown in FIG. 17.
The operation of interleaving the data from one unit to another is extremely complex if the data quantity to be interleaved differs from one angle to another. If there are many angles, the processing time is exponentially increased.
If bits are assigned so that the amounts of data of the respective angles in the interleaving units are the same at the state of output data following the encoding, as shown in FIG. 17A, it is unnecessary to insert stuffing dummy data shown in FIG. 17B, thus increasing the video rate at the angle positions. Thus, as shown in FIG. 17C, the same target quantity is assigned to each associated GOP at the time of bit assignment for the respective angle portions. Thus, it is desirable to assign the same bit quantity to the same seamless angle block.
FIG. 18 shows a processing example of the above-described seamless angle block.
For seamless reproduction on the side of the reproducing device, it suffices if control is performed so that the value of the residual buffer quantity of the VBV of the picture of the next start point will be larger than a predetermined value, such as VBV MAX*⅔.
FIG. 19A shows an example of encoding in which limitations are placed on the residual VBV buffer quantity, while FIG. 19B shows the residual buffer quantity when the encoding results are reproduced by a decoder (reproducing device).
In this manner, the residual buffer quantity in the decoder is necessarily larger than the calculated VBV value at the time of encoding. If, in the decoder, the buffer is full, there ceases the supply from the pickup to the buffer, so that there is no necessity of taking buffer overflow into consideration. Therefore, since there is no risk of buffer failure at the time of switching, seamless switching is assured if limitations are imposed in this manner at the time of encoding.
Also, since GOP-based switching is presupposed in the seamless angle, the first and the last VBV values of all GOPs are controlled to be not less than pre-set values, as shown in FIG. 20.
Since there are many limitations in the seamless angle block compared to the routine blocks, it is difficult to realize equivalent picture quality if the same bitrate is assigned. It is therefore desirable to assign bits preferentially to the seamless angle block.
However, if closed bit assignment is executed from one recording layer to another for video data containing seamless angle blocks, and which are recorded on plural recording layers, such as DVDs, there are occasions wherein a sufficient bitrate is not assigned to seamless angle blocks in certain recording layer(s). If sufficient picture quality is not realized after encoding, it becomes necessary to change the conditions and to perform the encoding operations from the outset.
In the encoding operation for preparing a disc of a package medium, such as DVD, the operations for respective parts, such as audio, video, subtitle or menu, are executed in parallel. Therefore, if it becomes necessary to correct the video encoding conditions, the encoding process other than the video encoding process, such as that for audio, for example, is affected by such correction. Thus, the encoding process other than the video encoding process needs to be performed again from the outset. That is, delay in judgment as to appropriateness of bit assignment leads to a large number of wasteful steps accompanying the re-encoding operations.
It is therefore an object of the present invention to provide a two-path encoding method and apparatus having a function such that, in assigning bits to compressed video data recorded on plural recording layers, it is evaluated whether or not bit assignment can be achieved so that no significant difference occurs in the picture quality between respective recording areas under given encoding conditions and, if the picture quality is not up to given encoding conditions, contents of change of encoding conditions providing an optimum or allowable variation quantity can be presented to an operator.
In one aspect, the present invention provides an encoding method for encoding a video material for recording in a plurality of recording areas, including a preliminary encoding step prior to ultimate encoding, an evaluation step for evaluating bit assignment as to allowability of the difference in picture quality between different recording areas under given encoding conditions and a presentation step for presenting an optimum range of bit assignment for which the picture quality difference between the different recording areas is allowable. The ultimate encoding is executed under the given encoding conditions or by the bit assignment of the presented optimum range.
In another aspect, the present invention provides an encoding method for encoding a material in which seamless angle blocks are contained in a plurality of recording areas, including a step of finding an average rate of the encoding difficulty from the sum of the number of bytes and the total number of frames, of each of which an upper limit accorded from one recording area to another is taken into account, a step of securing, from each recording area, the number of bytes assigned to the seamless angle block to be recorded in each recording area in accordance with the ratio of the number of frames which takes a weighting coefficient into account, and a step of assigning the residual number of bytes, obtained after deduction of the number of bytes assigned to the seamless angle blocks, in proportions of the sum of the encoding difficulty in which weighting coefficients are considered for blocks other than the seamless angle blocks.
In still another aspect, the present invention provides an encoding apparatus for encoding a video material for recording in a plurality of recording areas, including preliminary encoding means for doing preliminary encoding prior to ultimate encoding, evaluation means for evaluating bit assignment as to allowability of the difference in picture quality between different recording areas under given encoding conditions and presentation means for presenting an optimum range of bit assignment for which the picture quality difference between the different recording areas is allowable. The ultimate encoding is executed under the given encoding conditions or by the bit assignment of the presented optimum range.
In yet another aspect, the present invention provides an encoding method for encoding a material in which seamless angle blocks are contained in a plurality of recording areas thereof, including a step of finding an average rate of the encoded material from the sum of the number of bytes and the total number of frames, an upper limit of each of which accorded from one recording area to another is taken into account, a step of securing, from each recording area, the number of bytes assigned to the seamless angle block to be recorded in each recording area in accordance with the ratio of the number of frames which takes a weighting coefficient into account and a step of assigning the residual number of bytes, obtained on deduction of the number of bytes assigned to the seamless angle blocks, to blocks other than the seamless angle blocks in proportions of the sum of the encoding difficulty which takes a weighting coefficient into account.
In the above-described encoding method and apparatus of the present invention, if bit assignment is done for compressed video data recorded in plural recording areas, the difference in the picture quality between plural recording areas is hardly produced such that encoding conditions can be reconsidered or changed at an earlier time.
According to the present invention, the quantity of picture quality variation across plural recording layers or areas under the encoding conditions accorded by the supervisor prior to execution of the ultimate encoding can be predicted and evaluated so that the supervisor can be prompted to reconsider the conditions at an earlier time point and hence the number of encoding steps for disc formulation of the package medium such as DVD can be reduced.